Method and apparatus for head worn display with multiple exit pupils

ABSTRACT

A method for displaying an image viewable by an eye, the image being projected from a portable head worn display, comprises steps of: emitting a plurality of light beams of wavelengths that differ amongst the light beams; directing the plurality of light beams to a scanning mirror; modulating in intensity each one of the plurality of light beams in accordance with intensity information provided from the image, whereby the intensity is representative of a pixel value within the image; scanning the plurality of light beams in two distinct axes with the scanning mirror to form the image; and redirecting the plurality of light beams to the eye using a holographic optical element acting as a reflector of the light beams, whereby the redirecting is dependent on the wavelength of the light beam, to create for each light beam an exit pupil at the eye that is spatially separated from the exit pupils of the other light beams.

FIELD OF THE INVENTION

The present invention relates to head-worn displays (HWDs), inparticular, those systems that give the possibility to superimposevirtual images onto normal vision using eyewear with built in displaycapability.

BACKGROUND

The large-scale adoption of consumer mobile computing devices such assmartphones offers new possibilities for human interaction withcomputers and as well as the surrounding environment. Next generationmobile devices are expected to provide information by displaying it in adifferent manner than the current hand portable display screen. Advancesin projection display technologies are enabling near the eye displays,such as a pair of see through glasses with overlaid informationdisplayed directly to the user.

See-through displays have been used for decades for defenseapplications. For example, Jet fighter pilots have been using heads-updisplays (HUDs) in aircraft and helmet-mounted displays (HMDs) toprovide navigational and other critical information to the pilot inhis/her field of view. While projection technology is advancing, thereis still currently a difficult trade-off between field of view and thebulk and weight in see-through HWDs. In most cases a significant fieldof view (>30-40 degrees) requires bulky optics making their usagedifficult for many applications. Smaller field of view systems have beenintroduced with more acceptable form-factors, but the challenge remainsto create useful implementations of see-through displays withaesthetically pleasing form factors for a wide range of applications andeven everyday use.

One primary challenge in the design of HWDs is the expansion of theso-called eyebox of the display. The eyebox is an optical systemtolerance to the placement and movement of the wearer's eye. Thiscorresponds closely to the exit pupil of the optical system. Theconventional approach in HWDs is to expand the optical system's exitpupil by various means. However this usually leads to a more bulkyoptical system.

HWDs are often implemented using microdisplay panels, such as LCOS andOLED panel arrays, which are presented to the eye in a pupil forming ornon-pupil forming arrangement of imaging optics which allow the wearerto see a distant image of the microdisplay. Another but less commonapproach is retinal projection. Retinal projection uses a scanningelement to raster scan an image directly onto the user's retina. Retinalprojection displays originate with the scanning laser ophthalmoscope(SLO) developed in 1980. The technology was later developed into thevirtual retinal display, led by Tom Furness at the University ofWashington's HITLab in the 90s (Thomas A. Furness et al. “Display systemfor a head mounted viewing transparency” U.S. Pat. No. 5,162,828, filed1989), (Thomas A. Furness et al. “Virtual retinal dispay” U.S. Pat. No.5,467,104, filed 1992). Since then many HWD patents have been filedusing MEMS based scanning projectors, i.e. retinal displays. Ofparticular note are patents owned by the University of Washington andMicrovision (a spinoff of University of Washington) who led earlyefforts to commercialize the virtual retinal display in the mid-late90s. The majority of this work involved efforts to expand the exit pupilof the system, which is otherwise small due to the low étendue lasersource. The prevalent method found in patent literature is the use of adiffractive or diffusing screen to expand the beam, which is thenre-collimated before presenting it to the eye (Joel S. Kollin et al,“Virtual retinal display with expanded exit pupil” U.S. Pat. No.5,701,132, filed 1996). The drawback of this approach is that the beamexpansion optics creates added optical bulk with trade-offs similar toother conventional HWD approaches.

There have been methods to create multiple and/or steerable small exitpupils. These methods have used an array of lasers for multiple eyeboxlocations in conjunction with eye-tracking (M. Tidwell, “Virtual retinaldisplay with scanner array for generating multiple exit pupils”, U.S.Pat. No. 6,043,799, filed 1998), (M. Tidwell, “Scanned retinal displaywith exit pupil selected based on viewer's eye position,” U.S. Pat. No.6,204,829, filed 2000). Systems with steerable exit pupils based oneye-tracking have also been proposed (John R. Lewis et al., “Personaldisplay with vision tracking” U.S. Pat. No. 6,396,461, filed 1998).These systems relied on eye tracking and did not use a method to unifythe images produced by the multiple exit pupils.

There have been several HWD implementations using of Holographic OpticalElements (HOEs). Takahashi et al. have applied for a patent for a systemusing an HOE and Maxwellian view arrangement (Hideya Takahashi et al.,“Image Display Unit and Electronic Glasses”, U.S. patent applicationSer. No. 11/576,830, filed 2005), however the system in this patent doesnot appear to consider a laser scanning projector, but rather anexpanded beam passed through a spatial light modulator. In additionthere is no discussion of multiplexing or multiple exit pupils.

The concept of using a microdisplay in conjunction with a single layerhologram as a beam combiner is also known prior art—for example U.S.Pat. No. 3,940,204. From a related journal publication on the work, itwas described that aberrations were quite large in these systems. Thiswas largely due to the requirement for a large exit pupil of 12-15 mm atthe eye. This creates a larger aperture, “faster” optical system, whichis more difficult to control in terms of aberrations. The size of theprojector is also directly proportional to the numerical aperture of thebeam and the size of the exit pupil at the eye location.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus to display animage on a person's vision using a display built into eyewear.

In at least one embodiment, the present invention relates to a HWDcreated using several elements. 1) A scanning projection system thatconsists of a small étendue source—a laser or light emitting diode(LED), reflected from a resonant micro-electromechanical system (MEMS)scan mirror. For each position of the scan mirror a pixel may be formedon the retina through raster scanning. 2) A volume holographic opticalelement (HOE) used as a transflector, i.e., a transparent screen thatreflects light from a projection system, to redirect light to the eye.The transflector element performs two primary functions. It allowsambient light from the environment to pass through providing normalvision to the user. It also redirects scanned light from the projectorto the eye to provide a displayed image on the retina.

In at least one embodiment, the present invention relates to a methodfor placing a plurality of small exit pupils on the eye with controlover their locations. The combination of small and separated exit pupilscreates a larger effective eye-box. Alternatively, rather than fillingthe entire field of view, the exit pupils can be strategically placedfor particular view locations.

In at least one embodiment, the transflector is fabricated using areflective holographic recording technique, where two coherent laserbeams at a given wavelength of light and similar intensity interfere. Atthe given light wavelength, one of the two beams, called the referencebeam, is incident on the holographic material with the properties of thescanning projection system, i.e. with incidence angles and positionsmatching the projection setup. The second beam at the given lightwavelength, called the object beam, is incident on the opposite side ofthe holographic material to produce a reflection hologram. In oneembodiment, the object beam is passed through a large focusing lens toproduce a focus of the reflection hologram at the eye. Doing so placesthe exit pupil of the projection system at the center of the entrancepupil for the eye such that all parts of the projected image may beviewable for a given eye location and gaze-angle. In another embodiment,the exit pupil location can be placed not at the entrance pupil of theeye, but at the eye's center of rotation or in between the eye entrancepupil and center of eye rotation.

Further, in at least one embodiment, several separated wavelengths ofsimilar perceived color are used for holographic multiplexing to createmultiple exit pupil locations. By using multiple wavelengths with smallseparations for both the recording and read-out of the hologram, severalexit pupil locations can be created with similar colors. The exit pupillocations are created in the hologram writing setup by relay imaging anarray of point sources to the eye location, i.e. multiple object beamswritten simultaneously. In at least one embodiment, this is done usingoptical fiber facets in a precision arrangement acting as point sources.The point source arrangement of optical fibers is then relay imaged tothe eye location with (de)magnification. By projecting the light atsimilar wavelengths used for recording when the hologram is “read out”by the projector in the HWD, the multiple exit pupils are created. In atleast one embodiment, each of the narrowband wavelengths areindividually controlled and pre-processed in software before projectionto produce spatially shifted and distortion compensated imagescorresponding to their individual exit pupil location. In this way asingle image is created at the eye. In at least one embodiment, directmodulated lasers are beam combined to form the light source. In oneembodiment, the light beams are combined coaxially for presentation tothe HOE. In another embodiment the light beams are combined withdifferent angles that correspond to the spatial separations of theirrespective exit pupils. Alternatively in another embodiment multiple LEDlight sources are spectrally and spatially filtered, and beam combinedcoaxially or non-coaxially to form the light source. Further, in anotherembodiment, a broadband light source such as a white-light LED or laseris modulated by an array of spectrally selective modulators, such aselectro-absorption or acousto-optic modulators.

Further, in another embodiment, multiplexing red, green, and blue (RGB)light wavelengths for each exit pupil location creates a full color HWDwith multiple exit pupils. Doing so requires 3× the individuallycontrollable sources or light bands as the number of exit pupillocations. For example a color single exit pupil design requires 3individually controllable light bands: one red, one green, one blue. Adesign with two exit pupil locations requires 6 controllable wavelengthbands: two red, two green, and two blue. Similarly a design with 7 exitpupil locations would require 21 light bands with individual control.

In yet another embodiment, the multiple exit pupils are turned on andoff to reduce power consumption. Eye tracking is used to determine whichexit pupils should be used at any given moment.

In another embodiment, multiple exit pupils are created in a HWD thatuses a micropanel display element, rather than scanning mirror.Wavelength multiplexing in a HOE is used to separate the light ofdifferent wavelengths at the eye to create multiple exit pupils. Imagepre-processing of the various sources is used to align the apparentimages of the different exit pupil locations.

In another embodiment, multiple exit pupils are created in a scanningmirror HWD using angular multiplexing in the HOE transflector. Multiplescanning mirrors are used to create differences in angle of incidence atthe HOE, which redirects the light to multiple exit pupils. Imagepre-processing of the various sources is used to align the apparentimages of the different exit pupil locations.

In another embodiment, multiple contiguous field of view regions arecreated in a HWD that is non-pupil forming, meaning that projected lightis essentially collimated at the HOE transflector. The non-pupil formingapproach allows more easily for a large eyebox to be created, but can belimited in field of view by the HOE transflector. Therefore,multiplexing in the HOE is used to create multiple fields of view toenlarge the overall combined field of view. Image preprocessing of thevarious sources is used to align the apparent images of the differentfields of view.

In another embodiment, a scanning mirror HWD captures images of theretina and other parts of the eye by detecting the reflected signal fromthe eye in a confocal imaging arrangement. In one embodiment, thisreturn signal is used for eye tracking by correlating the detected imageto gaze angle. In another embodiment, tracking of the eye's position isdone by detecting and comparing the return signal intensity for aplurality of exit pupils, wherein comparison of the return signalindicates which exit pupil is best aligned to the eye.

In another embodiment, a broadband source is divided into multiplediscrete spectral emission bands in a scanning mirror HWD. Each of thediscrete spectral bands then creates an independent and spatiallyseparated exit pupil at the eye.

In another method of the invention, a narrowband diffusing element isused as the HWD transflector, and a spatial light modulator is used asthe light projection element, where the spatial light modulator is usedto phase conjugate the light's wavefront after the scatteringtransflector for a low aberration image at the eye.

In another embodiment, the HWD of this invention is used as a noninvasive method for health monitoring via the measurement ofphysiological parameters through the eye. For example, heart rate,glucose level, ocular pressure, level of stress, the nature or onset ofdiseases by imaging the retina at regular intervals.

The present invention applies to both monocular and binocularimplementations of a HWD. Unless otherwise noted the descriptions willcover the monocular arrangement with extension to a binoculararrangement requiring a replication of the optics and processing forboth eyes.

The present invention is not limited to HWDs. The method and devicestructure described can also be applied to head-up displays (HUDs)-seethrough display systems placed at a larger distance from the eye.

Accordingly, in a first aspect, the invention provides a method fordisplaying an image viewable by an eye, the image being projected from aportable head worn display. The method comprises steps of: emitting aplurality of light beams of wavelengths that differ amongst the lightbeams; directing the plurality of light beams to a scanning mirror;modulating in intensity each one of the plurality of light beams inaccordance with intensity information provided from the image, wherebythe intensity is representative of a pixel value within the image;scanning the plurality of light beams in two distinct axes with thescanning mirror to form the image; and redirecting the plurality oflight beams to the eye using a holographic optical element acting as areflector of the light beams, whereby the redirecting is dependent onthe wavelength of the light beam, to create for each light beam an exitpupil at the eye that is spatially separated from the exit pupils of theother light beams.

In a further preferred embodiment the step of emitting the plurality oflight beams further comprises creating a first bundle of a determinednumber of the plurality of light beams by selecting the correspondingwavelengths of those light beams to be comprised in a specific spectralband within a first given color as perceived by human vision, whereineach of the light beams in the first bundle is associated with its exitpupil that is spatially separated from the exit pupils of the otherlight beams of the first bundle.

In a further preferred embodiment the step of emitting the plurality oflight beams further comprises creating a second and a third bundle oflight beams, each bundle corresponding to a separate spectral bandwithin respectively a second given color and a third given color asperceived by human vision, wherein inside the second and the thirdbundle respectively the light beams are associated with the exit pupilsof the light beams of the first bundle, thereby creating for each of theexit pupils three light beams corresponding to the first, second andthird given colors for a full color image.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises applying an image registration anda distortion correction to the image for each of the light beams, toalign the displayed image produced by the plurality of light beams inaccordance to a location of the exit pupil for each light beam.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises spatially arranging the exit pupilsformed by the plurality of light beams to form an enlarged area in whichthe eye is aligned to the portable head worn display for viewing of theimage.

In a further preferred embodiment the step of directing the plurality oflight beams to the scanning mirror further comprises combining theplurality of light beams coaxially, both spatially and angularly, at thescanning mirror, whereby significant angular differences between thelight beams at the eye produced by positioning of the individual exitpupils are corrected by image processing.

In a second aspect the invention provides a method for displaying animage viewable by an eye, the image being projected from a portable headworn display. The method comprises steps of: emitting a plurality oflight beams of wavelengths that differ amongst the light beams;modulating in intensity each one of the plurality of light beams inaccordance with intensity information provided from the image, wherebythe intensity is representative of a pixel value within the image;redirecting the plurality of light beams to the eye using an opticalelement acting as a reflector of the light beams, whereby theredirecting is dependent on the wavelength of the light beam, to createfor each light beam an exit pupil at the eye.

In a further preferred embodiment the step of emitting the plurality oflight beams further comprises creating a first bundle of a determinednumber of the plurality of light beams by selecting the correspondingwavelengths of those light beams to be comprised in a specific spectralband within a first given color as perceived by human vision, whereineach of the light beams in the first bundle is associated with its exitpupil that is spatially separated from the exit pupils of the otherlight beams of the first bundle.

In a further preferred embodiment the step of emitting the plurality oflight beams further comprises creating a second and a third bundle oflight beams, each bundle corresponding to a separate spectral bandwithin respectively a second given color and a third given color asperceived by human vision, wherein inside the second and the thirdbundle respectively the light beams are associated with the exit pupilsof the light beams of the first bundle, thereby creating for each of theexit pupils three light beams corresponding to the first, second andthird given colors for a full color image.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises applying an image registration anda distortion correction to the image for each of the light beams, toalign the displayed image produced by the plurality of light beams inaccordance to a location of the exit pupil for each light beam.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises spatially arranging the exit pupilsformed by the plurality of light beams to form an enlarged area in whichthe eye is aligned to the portable head worn display for viewing of theimage.

In a further preferred embodiment the step of redirecting the pluralityof light beams, the optical element is a holographic optical element.

In a third aspect the invention provides a method of producing anoptical element for use in the method for displaying an image viewableby an eye, comprising recording a holographic optical element with aplurality of hologram writing lasers closely matched to the wavelengthsof the plurality of light beams, and whereby the beams of each of thewriting lasers are arranged spatially in a hologram recording setup tomatch the spatial orientation of the exit pupils to be subsequentlycreated by the portable head worn display.

In a further preferred embodiment the beams of each of the writinglasers are arranged spatially by means of optical fibers.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises deactivating selected ones of theplurality of light beams associated to each of the exit pupils, incorrespondence to the eye's position at a given moment, whereby eyetracking information is used to deactivate misaligned exit pupils toreduce device power consumption.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises spatially arranging the exit pupilsformed by the plurality of light beams, whereby each individual lightbeam forms a plurality of spatially separated exit pupils, to createmultiple regions of interest that are not viewed simultaneously by theeye, each with a subset field of view and associated plurality of exitpupils within a larger overall field of view.

In a further preferred embodiment the step of modulating in intensity ofeach one of the plurality of light beams comprises projecting from apanel microdisplay.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises directing the plurality of lightbeams to a scanning mirror; and scanning the plurality of light beams intwo distinct axes with the scanning mirror to form the image.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises combining the plurality of lightbeams coaxially, both spatially and angularly, whereby significantangular differences between the light beams at the eye produced bypositioning of the individual exit pupils are then corrected by imageprocessing.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises combining the plurality of lightbeams with angular differences between the light beams such that theangular content of one of the light beams at the exit pupils issubstantially similar to the angular content of any other one of thelight beams, thereby reducing the requirements for image processing,whereby remaining angular differences between the light beams at the eyeproduced by positioning of the individual exit pupils are then correctedby image processing.

In a further preferred embodiment the plurality light beams are combinedwith angular differences between the light beams by means of atelecentric lens which combines the light beams at the exit pupil of thetelecentric lens, whereby a two-dimensional arrangement of a pluralityof light sources that emit the light beams of wavelengths that differamongst the light beams are collimated and combined.

In a further preferred embodiment the plurality of light sources thatemit the light beams of wavelengths that differ amongst the light beamsare spatially and spectrally filtered by a combination of a furthertelecentric lens, diffracting optical element and apertures for thelight beams.

In a further preferred embodiment the diffracting optical element is oneof the following list: a diffraction grating, a volume holographicelement.

In a fourth aspect the invention provides a method for displaying animage viewable by an eye, the image being projected from a portable headworn display. The method comprises steps of: emitting a plurality oflight beams; directing each of the plurality of light beams to acorresponding spatially separated scanning mirror; modulating inintensity each one of the plurality of light beams in accordance withintensity information provided from the image, whereby the intensity isrepresentative of a pixel value within the image; scanning each one ofthe plurality of light beams in two distinct axes with the correspondingone of the plurality of spatially separated scanning mirrors to form theimage; and redirecting the plurality of light beams to the eye using anoptical element acting as a reflector of the light beams, whereby theredirecting is dependent on the incidence angle of the light beam on theoptical element, to create for each light beam an exit pupil at the eyethat is spatially separated from the exit pupils of the other lightbeams.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises applying an image registration anda distortion correction to the image for each of the light beams, toalign the displayed image produced by the plurality of light beams inaccordance to a location of the exit pupil for each light beam.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises spatially arranging the exit pupilsformed by the plurality of light beams to form an enlarged area in whichthe eye is aligned to the portable head worn display for viewing of theimage.

In a further preferred embodiment the step of emitting the plurality oflight beams further, three light beams of separate visible wavelengthsare directed to each scanning mirror and combined to form one of theexit pupils, thereby creating for each of the exit pupils three lightbeams for a full color image.

In a further preferred embodiment the step of redirecting the pluralityof light beams, the optical element is a holographic optical element.

In a fifth aspect the invention provides a method of producing anoptical element for use in the method for displaying an image viewableby an eye further comprises. The method further comprises: recording aholographic optical element with a plurality of hologram writing lasersclosely matched to the wavelengths of the plurality of light beams, andwhereby the beams of each of the writing lasers are arranged spatiallyin a hologram recording setup to match the spatial and angularorientation of the exit pupils and projection source points to besubsequently created by the portable head worn display.

In a further preferred embodiment the beams of each of the writinglasers are arranged spatially by means of optical fibers.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises deactivating selected ones of theplurality of light beams associated to each of the exit pupils, incorrespondence to the eye's position at a given moment, whereby eyetracking information is used to deactivate misaligned exit pupils toreduce device power consumption.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises arranging the exit pupils formed bythe plurality of light beams, whereby each individual light beam forms aplurality of spatially separated exit pupils, to create multiple regionsof interest that are not viewed simultaneously by the eye, each with asubset field of view and associated plurality of exit pupils within alarger overall field of view.

In a sixth aspect the invention provides a method for displaying animage viewable by an eye, the image being projected from a portable headworn display. The method comprises steps of: emitting a plurality oflight beams; modulating in intensity each one of the plurality of lightbeams in accordance with intensity information provided from the image,whereby the intensity is representative of a pixel value within theimage; redirecting the plurality of substantially collimated light beamsto the eye without creating an optical exit pupil at the eye using anoptical element acting as a reflector of the light beams, whereby theredirecting is dependent on the angle of incidence and wavelength of thelight beam, creating a plurality of subset fields of view which make upan overall field of view in ensemble at the eye.

In a further preferred embodiment in the step of redirecting theplurality of light beams, the optical element is a holographic opticalelement.

In a seventh preferred embodiment the invention provides a method ofproducing an optical element for use in the method for displaying animage viewable by an eye further comprising recording a holographicoptical element with a plurality of hologram writing lasers closelymatched to the wavelengths of the plurality of light beams, and wherebythe beams of each of the writing lasers are arranged spatially in ahologram recording setup to match the spatial and angular orientation ofthe subset fields of view to be subsequently created by the portablehead worn display.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises applying an image registration anda distortion correction to the image for each of the light beams, toalign the images of the subset fields of view to form in a contiguousfashion, the overall field of view in ensemble.

In a further preferred embodiment in the step of emitting the pluralityof light beams, three light beams of separate visible wavelengths persubset field of view, are combined thereby creating for each of thesubset fields of view, three light beams for a full color image.

In an eighth aspect the invention provides a method for obtainingphysiological information from the eye by projecting and capturing animage from a portable head worn display. The method comprises the stepsof: emitting a plurality of light beams of wavelengths that differamongst the light beams; focusing the plurality of light beams through alens onto a pinhole aperture; directing the plurality of light beamsfrom the pinhole aperture to a scanning mirror; modulating in intensityeach one of the plurality of light beams in accordance with intensityinformation provided from the image, whereby the intensity isrepresentative of a pixel value within the image; scanning the pluralityof light beams in two distinct axes with the scanning mirror to form theimage; redirecting the plurality of light beams to the eye using anoptical element on the spectacle lens acting as a reflector of the lightbeams, whereby the redirecting is dependent on the wavelength and angleof the light beam, to create for each light beam an exit pupil at theeye that is spatially separated from the exit pupils of the other lightbeams; focusing the redirected plurality of light beams onto a surfaceof the eye; reflecting the focused plurality of light from said surfaceof the eye back through the system to the pinhole aperture; anddirecting the reflected plurality of light beams through a beamsplitting element to a detector, whereby the intensity is representativeof the confocal image of the surface of the eye where the plurality oflight beams were focused.

In a further preferred embodiment the pinhole aperture is replaced by anoptical fiber used to transport the light to a different location.

In a further preferred embodiment light is reflected back at the samewavelengths as the plurality of light beams via scattering from theeye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via fluorescencefrom the eye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via Ramanscattering from the eye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via non-linearphenomenon at the eye's surface.

In a further preferred embodiment the optical element is a volumeholographic optical element.

In a ninth aspect the invention provides a method for obtainingphysiological information from the eye by projecting and capturing animage from a portable head worn display. The method comprises steps of:emitting a plurality of light beams of wavelengths that differ amongstthe light beams; focusing the plurality of light beams through a lensonto a single mode core of a multimode dual cladding optical fiber;directing the plurality of light beams from the multimode dual claddingoptical fiber to a scanning mirror; modulating in intensity each one ofthe plurality of light beams in accordance with intensity informationprovided from the image, whereby the intensity is representative of apixel value within the image; scanning the plurality of light beams intwo distinct axes with the scanning mirror to form the image;redirecting the plurality of light beams to the eye using an opticalelement on the spectacle lens acting as a reflector of the light beams,whereby the redirecting is dependent on the wavelength and angle of thelight beam, to create for each light beam an exit pupil at the eye thatis spatially separated from the exit pupils of the other light beams;focusing the redirected plurality of light beams onto a surface of theeye; reflecting the plurality of light beams from said surface of theeye back through the system to the multimode core of said multimode dualcladding optical fiber; and directing the reflected plurality of lightbeams through a beam splitting element to a detector, whereby theintensity is representative of the confocal image of the surface of theeye where the plurality of light beams were focused.

In a further preferred embodiment light is reflected back at the samewavelengths as the plurality of light beams via scattering from theeye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via fluorescencefrom the eye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via Ramanscattering from the eye's surface.

In a further preferred embodiment light is reflected back at shiftedwavelengths as compared to the plurality of light beams via non-linearphenomenon at the eye's surface.

In a further preferred embodiment the optical element is one of thefollowing list: a kinoform diffractive optical element, a curvedreflective element with a frequency selective response.

In a further preferred embodiment the method for obtaining physiologicalinformation from the eye further comprises focusing the plurality oflight beams to different depths of the eye, whereby the wavelengths ofthe light beams determine which structures of the eye are imaged.

In a further preferred embodiment invisible infrared light is used forthe confocal measurement so as not to disturb the visible function ofthe head worn display.

In a further preferred embodiment the beams are separated at thedetector by filters which is any one from the following list:interference type, dichroic, holographic.

In a further preferred embodiment the method for obtaining physiologicalinformation from the eye further comprises: capturing a plurality ofgaze specific images of the eye for eye tracking calibration; processingthe plurality of gaze specific images of the eye for feature extraction;forming a database correlating gaze positions to the extracted features;capturing a new gaze specific image of the eye for gaze determination;correlating the features of this image against images in the database;and classifying the image by correlation to a particular gaze angle, foreye tracking in real time.

In a further preferred embodiment the method for obtaining physiologicalinformation from the eye further comprises: capturing the reflectedintensity from the plurality of light beams that make up the pluralityof exit pupils at the eye, whereby the eye's gaze position is correlatedto the relative intensity of the plurality of beams that make up thespatially separated exit pupils at the eye.

In a tenth aspect the invention provides a method for displaying animage viewable by an eye, the image being projected from a portable headworn display. The method comprises steps of: emitting at least one lightbeam of broad spectrum; slicing the spectrum of the at least one lightbeam into a plurality of discrete spectral emission bands each separatedby a spectral zone of no light; directing the plurality of light beamsto a scanning mirror; modulating in intensity each one of the at leastone light beams in accordance with intensity information provided fromthe image, whereby the intensity is representative of a pixel valuewithin the image; scanning the at least one light beams in two distinctaxes with the scanning mirror to form the image; and redirecting the atleast one light beams to the eye using a holographic optical elementacting as a reflector of the light beams, whereby the redirecting isdependent on the wavelength content and angle of the light beam, tocreate for each of the discrete spectral emission bands an exit pupil atthe eye that is spatially separated from the exit pupils of the otherdiscrete spectral emission bands.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises deflecting the at least one lightbeam after the scanning mirror by a dispersive optical element, wherebythe dispersive optical element separates the at least one light beamangularly into a plurality of light beams corresponding to the number ofdiscrete spectral emission bands.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises emitting three light beams withseparated spectral bands within each of the red, green and blue colorranges, such that the combination of one light beam from each colorrange creates a perceived color hue with the hue dependent on therelative strength of the three combined light beams.

In a further preferred embodiment the method for displaying an imageviewable by an eye further comprises spatially arranging the pluralityof exit pupils formed by the plurality of light beams to form anenlarged area in which the eye can be aligned to the optical system forviewing of the displayed image.

In an eleventh aspect the invention provides a method for displaying animage viewable by an eye, the image being projected from a portable headworn display. The method comprises steps of: emitting at least one lightbeam from a coherent source; directing the at least one light beam to aspatial light modulator having a phase pattern which provides a firstwavefront; redirecting the at least one light beam to the eye using adiffuse scattering reflector on the spectacle lens, whereby the firstwavefront is reflected by the diffuse scattering reflector providing asecond wavefront entering the eye and forming a low aberration image onthe retina.

In a further preferred embodiment the coherent light source is a surfaceemitting laser (VCSEL), a point source LED or a laser diode.

In a further preferred embodiment the spectacle lens is composed of afirst and a second transparent optically joined elements, wherein thefirst element has a first index of refraction has one side which isscattering with visible light and a second side which is smooth and nonscattering and on which a reflective coating is deposited, and thesecond element has a second index of refraction equal to said firstindex of refraction and having a smooth and non scattering side.

In a further preferred embodiment the spatial light modulator is a phaseonly modulator, an amplitude only modulator or both.

In a twelfth aspect the invention provides a method for obtainingphysiological information from the eye by projecting and capturing animage viewable by an eye, the image being projected from a portable headworn display the method comprises steps of: emitting at least one lightbeam from a coherent source; directing the at least one light beams to aspatial light modulator having a phase pattern which provides a firstwavefront; redirecting the at least one light beams to the eye using adiffuse scattering reflector on the spectacle lens, whereby the firstwavefront is reflected by the diffuse scattering reflector providing asecond wavefront entering the eye and forming a low aberration spot on asurface of the eye; scanning the spot on the retina by providing anappropriate phase pattern to the SLM; and retrieving the diffusereflected light by the retina in a confocal manner to form an image ofsaid surface.

In a further preferred embodiment said surface is the retina.

In a thirteenth aspect, the invention provides a system for displayingan image viewable by an eye, the image being projected from a portablehead worn display. The system comprises: a multiple exit pupil head worndisplay system for implementing the method for displaying an imageviewable by an eye; and a front mounted camera that captures a scene andprovides a processed image of the scene to the head worn display.

In a further preferred embodiment the processed image can be any of (a)a zoomed imaged, (b) edge-enhanced image (c) contrast enhanced image (d)a distorted image or a combination of (a) to (d).

In a further preferred embodiment the eye has a loss of light receptorsin the fovea such as an eye suffering from age related maculardegeneration.

In a further preferred embodiment the processed image is displayed inthe periphery around the fovea of the eye.

In a further preferred embodiment the eye has a loss of light receptorsin the periphery around the fovea.

In a further preferred embodiment the processed image is displayed inthe fovea.

In a fourteenth aspect, the invention provides a device for redirectingan image projected from a portable head worn display to the eye. Thedevice comprises: embedded small kinoform mirrors in a transparentthermo polymer matrix to locally redirect at least one incident lightbeams to the eye; and a thin film reflective coating on the kinoformmirrors which is spectrally and angularly selective to allow for the atleast one light beam from the projector to be separated into multipleexit pupils at the eye, while simultaneously allowing substantialambient light from the surroundings to pass through to the eye.

In a fifteenth aspect, the invention provides a system for displayingbinocular images to two eyes, the images being projected from twoportable head worn displays. The system comprises: multiple exit pupilprojection modules on each side of a pair of spectacles; multiple exitpupil holographic reflectors on both the spectacle lenses; aneyetracking system for both eyes based on reflected light from the eye;a front facing camera to capture the scene in front of the user; andthree-dimensional images produced by changes to the binocular imagesbased on information from the eye tracking system.

In a sixteenth aspect, the invention provides a method for creating andmaintaining alignment between a projector and a spectacle lensholographic optical element in a multiple exit pupil head worn display.The method comprises steps of: aligning the projector and theholographic optical element on a rigid material connection structurallymaintaining positions and angle between the optical elements; andattaching and positioning the rigid structure on a pair of conventionalspectacles.

In a further preferred embodiment the conventional spectacles arenonprescription glasses or sunglasses; or, prescription glasses orsunglasses.

In a further preferred embodiment the rigid material connection isplaced and attached on the inside, or the outside of the conventionalspectacles.

These and other aspects of the invention will be explained further inthe following detailed description in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 0.1 is a drawing showing a device from prior art;

FIG. 1A is a drawing of a hardwired monocular head worn displayaccording to an embodiment of this invention;

FIG. 1B is a drawing of a wireless monocular head worn display accordingto an embodiment of this invention;

FIG. 2A is an optical schematic of the scanned retinal display alignedto the eye with a single exit pupil according to an embodiment of thisinvention;

FIG. 2B is an optical schematic of FIG. 2A, with the eye rotated andmisaligned to the single exit pupil;

FIG. 3A is an optical schematic of the scanned retinal display alignedto the eye with one of two exit pupils formed by wavelength multiplexingaccording to an embodiment of this invention;

FIG. 3B is an optical schematic of FIG. 3A with the eye rotated andaligned to the second of two exit pupils formed by wavelengthmultiplexing;

FIG. 4A is an illustration of the projected image with preprocessingshift between two image components related to two exit pupils;

FIG. 4B is an optical schematic of the scanned retinal display with twoidentical pixels for two different exit pupils projected on thetransflector merging into one single pixel on the retina;

FIG. 4C is an illustration of the simultaneous perceived image on theretina from two exit pupils with image preprocessing shown in FIG. 4A toalign the images according to an embodiment of this invention;

FIG. 4D is a demonstration of the enlarged eyebox and imagepre-processing where pictures are obtained with a rotating cameracombined with a focusing lens mimicking an artificial eye;

FIG. 5 is an optical schematic of the scanned retinal display with twoexit pupils and an associated illustration of the exit pupil arrangementand eyebox overlap according to an embodiment of this invention;

FIG. 6 is an illustration of three examples of exit pupil arrangementsand eye-box overlaps according to three different embodiments of thisinvention;

FIG. 7 is an illustration of the wavelength selectivity of a volumeholographic element according to an embodiment of this invention;

FIG. 8 is an optical schematic of the simultaneous hologram writingsetup for three exit pupils using three different center wavelengthsaccording to an embodiment of this invention;

FIG. 9A is an optical schematic of a coaxial spectral beam combinerusing a diffractive element according to an embodiment of thisinvention;

FIG. 9B is an optical schematic of a coaxial spectral beam combinerusing dichroic beamsplitters according to an embodiment of thisinvention;

FIG. 9C is an optical schematic of a coaxial spectral beam combinerusing a volume holographic element according to an embodiment of thisinvention;

FIG. 9D is an optical schematic of a coaxial spectral beam combinerusing a volume holographic element and external cavity according to anembodiment of this invention;

FIG. 10 is an optical schematic of the scanned retinal display on aneyewear frame using angular separation of the light beams to form twoexit pupils from two different wavelength sources according to anembodiment of this invention;

FIG. 11 is an optical schematic of the scanning projection module withnon-coaxially combined beams according to an embodiment of thisinvention;

FIG. 12 illustrates four examples of light source arrangements ofdifferent wavelengths to form spatially separated exit pupils andnon-coaxially combined beams according to four different embodiments ofthis invention;

FIG. 13A is an illustration of a light source with red, green and blueemitters at each of three locations to form spatially separated exitpupils and non-coaxially combined beams according to an embodiment ofthis invention;

FIG. 13B is an illustration of the apparent image at the eye from thelight source of FIG. 13A, without processing (left) and withpreprocesseing to align the images (right).

FIG. 14 is an optical schematic of the scanning projection module withnon-coaxially combined beams from red, green and blue sources accordingto an embodiment of this invention;

FIG. 15 is an optical schematic of the scanning projection module withnon-coaxially combined beams with additional spatial and spectralfiltering in reflection according to an embodiment of this invention;

FIG. 16 is a drawing of a monocular scanned beam head worn display usingthe projection module of FIG. 15 according to an embodiment of thisinvention;

FIG. 17 is a drawing of a binocular scanned beam head worn display usingthe projection module of FIG. 15 according to an embodiment of thisinvention;

FIG. 18 is an optical schematic of the scanning projection module withnon-coaxially combined beams with additional spatial and spectralfiltering in transmission according to an embodiment of this invention;

FIG. 19 is an optical schematic of a scanning projection module withnon-coaxially combined beams using a holographic filtering elementaccording to an embodiment of this invention;

FIG. 20 is an optical schematic of head worn display based on projectionof a micropanel display, aligned to the eye with one of three exitpupils formed by wavelength multiplexing according to an embodiment ofthis invention;

FIG. 21A is an optical schematic of the scanned retinal display with theeye aligned to one of two separated exit pupils with separated field ofview created with a single wavelength according to an embodiment of thisinvention;

FIG. 21B is an optical schematic of the scanned retinal display with theeye aligned to the second of two separated exit pupils with separatedfields of view created with a single wavelength according to anembodiment of this invention;

FIG. 21C is an optical schematic of the scanned retinal display with theeye aligned to one of two separated exit pupils with slightlyoverlapping fields of view created with two wavelengths according to anembodiment of this invention;

FIG. 21D is an optical schematic of the scanned retinal display with theeye aligned to one of two separated exit pupils with significantlyoverlapping fields of view created with two wavelengths according to anembodiment of this invention;

FIG. 21E is an optical schematic of the scanned retinal display with theeye aligned to one of two separated exit pupils with separated fields ofview created with two angularly separated sources according to anembodiment of this invention;

FIG. 21F is an optical schematic of the scanned retinal display with theeye aligned to one of two separated exit pupils with separated fields ofview created with a single source that is split and separated in angleaccording to an embodiment of this invention;

FIG. 22A is a top view optical schematic of the scanned retinal displaywith two scanning mirrors to create two separated exit pupils;

FIG. 22B is a side view optical schematic of the scanned retinal displaywith two scanning mirrors to create two separated exit pupils;

FIG. 23 is an optical schematic of the simultaneous hologram writingsetup for three exit pupils using angular multiplexing with one centerwavelength according to an embodiment of this invention;

FIG. 24A is an optical schematic of the simultaneous hologram writingmethod to create three distinct fields of view for a non-pupil forminghead worn display using angular multiplexing;

FIG. 24B is an illustration of the diffraction efficiency anddiffraction angle for two angle-multiplexed holograms for a non-pupilforming head worn display;

FIG. 25 is an optical schematic of the simultaneous hologram writingmethod to create three distinct fields of view for a non-pupil forminghead worn display using wavelength multiplexing;

FIG. 26 is a drawing of a scanning projection monocular head worndisplay according to an embodiment of this invention;

FIG. 27 is an optical schematic of a scanning projection head worndisplay including confocal imaging of the retina according to anembodiment of this invention;

FIG. 28 is an optical schematic of a scanning projection head worndisplay including confocal imaging of arbitrary eye surfaces accordingto an embodiment of this invention;

FIG. 29 is an optical schematic of a scanning projection head worndisplay including confocal imaging of the eye using an additionalwavelength separated with a diffraction grating according to anembodiment of this invention;

FIG. 30 is an optical schematic of a scanning projection head worndisplay including confocal imaging of the eye using an additionalwavelength separated with a dichroic beamsplitter according to anembodiment of this invention;

FIG. 31 is an image of a retina from an ophthalmoscope;

FIG. 32 is a set of three images of a single retina for three differentgaze directions;

FIG. 33 is a block diagram of eye tracking method according to anembodiment of this invention;

FIG. 34 is an illustration of the eye tracking method using relativeconfocal intensities from different exit pupils according to anembodiment of this invention;

FIG. 35 is a drawing of a scanning projection monocular head worndisplay including a camera according to an embodiment of this invention;

FIG. 36 is an illustration of the method to display data gathered fromthe camera on a head worn display according to an embodiment of thisinvention;

FIG. 37 is an optical schematic of a transflector using embeddednarrowband kinoform mirrors in a transparent material according to anembodiment of this invention;

FIG. 38 is an optical schematic of a scanning projection head worndisplay using a split spectrum source and wavelength multiplexing in aholographic transflector to create multiple exit pupils according to anembodiment of this invention;

FIG. 39 is an optical schematic of a scanning projection head worndisplay using a split spectrum source, a diffraction grating andwavelength multiplexing in a holographic transflector to create multipleexit pupils according to an embodiment of this invention;

FIG. 40 is a drawing of a scanning projection binocular head worndisplay including a camera according to an embodiment of this invention;

FIG. 41 is a drawing of a scanning projection binocular head worndisplay attached to the inside of a rigid eyewear frame according to anembodiment of this invention;

FIG. 42 is a drawing of a scanning projection binocular head worndisplay attached to the outside of a rigid eyewear frame according to anembodiment of this invention; and

FIG. 43 is an optical schematic of a scanning projection head worndisplay including confocal imaging of the retina with a double cladfiber according to an embodiment of this invention.

DETAILED DESCRIPTION

The techniques, apparatus, materials and systems as described in thisspecification can be used to implement a HWD based on a scanningprojector and holographic transflector, and can also be applied tohead-up displays (HUDs)—see through display systems placed at a largerdistance from the eye.

In at least one embodiment of the invention a head worn display createsa scanned image directly on the user's retina using a scanning mirrorprojector. The exit pupil of the scanning mirror is placed at theentrance pupil of the eye using a transflector element, which can be,but is not limited to a holographic optical element (HOE). In additionto reflecting the display light toward the eye, the transflector alsoacts to efficiently transmit light from the environment to the eye,allowing for the display to be added to natural vision. This is oftenreferred to as “augmented reality” or sometimes “mixed reality”.Additionally, the described invention allows for an effectively expandedeyebox by multiplexing the HOE to create multiple exit pupils arrangedto mimic an enlarged eyebox.

FIG. 1A and FIG. 1B show two possible embodiments of the invention. FIG.1A is a drawing of a possible embodiment of the present inventionshowing simple and lightweight eyewear integrated with the scanningdisplay. A layer of holographic material is coated onto one of thelenses of the glasses 101. In at least one embodiment, the HOE can be aphotopolymer coated on the lens and subsequently holographicallyrecorded or in another embodiment, the photopolymer can be embedded into(i.e., sandwiched) between two surfaces of the spectacle lens. Theholographic material then acts to redirect the display light into theeye, while transmitting the light from the environment. In one arm ofthe eyewear near the temple of the user, a scanning mirror or panelmicrodisplay projector 102 projects the image onto the holographictransflector 101. In one embodiment the light source, power electronicsand driver electronics are separately or altogether moved off the headin a small box 103 connected to the eyewear through a patch cord 104which can be detached at the glasses by a removable connector 105.Moving these components off the head has the advantage of allowing theeyewear to be simple, lightweight, discrete, cosmetically attractive andsocially acceptable. Further, the eyewear can be disconnected from theseparate module allowing a user to indicate to others whether or not thepersonal display device is active. This is an attractive feature in asocial context considering that a consumer head worn display will likelyalso include sound and video recording devices that may interfere withpersonal privacy and social interaction. In other embodiments with tightintegration of the components, the light sources, power and drivercomponents could alternatively be placed completely within the eyewear.FIG. 1B is a drawing of another possible embodiment where the lightsource, power electronics and driver electronics are containedcompletely within the eyewear 106. Further, in other embodiments lightcan be projected into both eyes in a binocular fashion using two sets ofprojectors and HOEs on both spectacle lenses.

In at least one embodiment, light travels through the optical system asdepicted in FIG. 2A. FIG. 2A is a schematic of the optical arrangementof a scanning projection based head worn display for a single exit pupilof the scanning mirror 212. Light of at least one wavelength 206 isincident on a scanning mirror 205, which scans the light beam in angle207 with intensity modulated to correspond to a digital image. Theholographic transflector 204, which in at least one embodiment includesa recorded holographic material 201 sandwiched between two outerprotective layers 202 and 203, reflects the projected light 207 intodiffracted light 208 toward the center of the eye entrance pupil 210 ofthe forward gazing eye 209. Here the eye line of sight 211 is aligned tothe center of the exit pupil of the scanning mirror 212. The limitationwith this arrangement is in the rotation tolerance of the eye. FIG. 2Bshows that an eye can rotate out of the view of the single exit pupil.As such, a single exit pupil arrangement is suitable for small field ofview displays where the user aligns their eye to view the displayedimage. In FIG. 2B, the eye line of sight 211 is misaligned to the singleexit pupil 212. The holographic transflector 204 reflects the projectedlight toward the center of the entrance pupil of the forward gazing eyebut mismatch between the eye's entrance pupil 210 and the exit pupil 212prevents the image from being seen.

In at least one embodiment of the invention, multiple exit pupilseffectively expand the system's eyebox. FIG. 3A and FIG. 3B show anexample of a multiple exit pupil arrangement, with two exit pupilscreated at two spatially separated positions near the eye. In FIG. 3Aand FIG. 3B two exit pupils are shown for simplicity, but additionalexit pupils can be used in a two-dimensional arrangement. Multiple exitpupils can be used to create a larger effective eyebox for the eye, andallow for large FOV images to be scanned by the eye. In at least oneembodiment, multiplexing the holographic element with differentwavelengths of light creates the multiple exit pupils. Volumeholographic elements exhibit “selectivity” meaning that the hologramwritten for one wavelength of light and angle of incidence isindependent of another hologram written at a sufficiently differentwavelength or incidence angle. In this way, sources of different centerwavelength or incidence angle can be used to produce multipleindependent exit pupils without crosstalk thus producing an enlargedeffective eyebox. Further, the multiple exit pupils can be created withseparated wavelengths of similar color in terms of human perception. Forexample, several exit pupil locations can be created using severaldifferent red light sources with sufficiently different centerwavelengths. The required separation depends on the spectral selectivityof the HOE to prevent crosstalk. FIG. 3A is a schematic of the opticalarrangement for two exit pupils 212 and 304 along the vertical axiscreated by holographic multiplexing of two light beams 206 and 301 ofdifferent wavelengths. These light beams are reflected by the scanningmirror 205, which scans the light beam in angles 207 and 302 withintensity modulated to correspond to a digital image The holographictransflector 204 reflects the projected light 207 and 302 intodiffracted light 208 and 303 toward two exit pupil locations 212 and 304at the plane of the eye, providing a tolerance to eye rotation. Here theeye line of sight 211 is aligned 211 to the center of the central exitpupil 212. FIG. 3B is a schematic of the optical arrangement for twoexit pupils along the vertical axis with the eye line of sight 211rotated but still capturing light from one of the exit pupils 304.

When multiple exit pupils are created with the holographic methoddescribed in this invention, light from a particular position of thescanning mirror, equivalently a pixel of the image, will appear on theretina at different positions for the different wavelengthscorresponding to different angular content of the light of the differentexit pupils. In at least one embodiment shown in FIG. 4, this differenceis corrected by preprocessing the images that are projected for eachwavelength corresponding to each individual exit pupil. When theindividual light sources are independently controlled, each position ofthe scanning mirror can project independent pixel data for the differentwavelength sources corresponding to the shifted positions of the exitpupils. As shown in FIG. 4A, the required image processing for laterallyshifted exit pupils is primarily a relative image shift between theprojected images for each of the different wavelengths sources.Additional geometric corrections may also be applied, for example tocorrect for distortion. When properly adjusted, the images from thedifferent wavelengths and exit pupil locations will coincide to form asingle image on the retina. FIG. 4 is an illustration of thepre-processing shifts and image registration necessary to align on theretina the images of the different exit pupils. In this example, FIG. 4Ashows two images 401 and 402 from two different exit pupils arrangedvertically. By projecting images 401 and 402 shifted from each other asin FIG. 4A, a single apparent image 403 is produced on the retina, asshown in FIG. 4C. Observing FIG. 4B, pixels 404 and 405 content theinformation of the same image pixel for each exit pupil. By projectingpixels 404 and 405 on the transflector with a separation distancesimilar to the separation distance of the two corresponding exit pupils,pixels 404 and 405 merge into one single pixel 406 on the retina. Realsystems can have additional shifts across 2-dimensions. Real systems mayalso benefit from further pre-processing of the images to correctnonlinearities such as distortion to improve alignment of the images. Ademonstration of the enlarged eyebox and image pre-processing is shownin FIG. 4D. The image 407 with a single wavelength componentcorresponding to a single exit pupil was projected onto thetransflector, which in turn reflected the image back into a cameracombined with a focusing lens mimicking an artificial eye. Picture 408was taken with the camera aligned with the single exit pupil, whereaspicture 409 was taken with the camera partly misaligned with the singleexit pupil and picture 410 was taken with the camera further misalignedwith the single exit pupil. Picture 408, 409 and 410 show that theprojected image disappears as the camera is misaligned with the singleexit pupil. The image 411 with a different wavelength component thanimage 407 corresponding to another exit pupil than the one obtained fromimage 407 was projected together with image 407 onto the transflector,which in turn reflected the image back into a camera combined with afocusing lens mimicking an artificial eye. Picture 412 was taken withthe camera aligned with the exit pupil of image 407, whereas picture 413was taken with the camera partly misaligned with the exit pupil ofimages 407 and 408. Picture 414 was taken with the camera furthermisaligned with the single exit pupil but aligned with the exit pupil ofimage 408. Picture 412, 413 and 414 demonstrate that the eyebox can beenlarged through the use of multiple exit pupils.

FIG. 5 is an illustration of how the individual exit pupils are relatedto the “eyebox” of the head worn display. In this example the system ofFIG. 3A and FIG. 3B is shown with two exit pupils. Each exit pupil 212and 304 creates a rotation tolerance for the eye, where the image can beviewed while light enters the eye's entrance pupil 210. The size of eachindividual eyebox 501 and 503 is approximately equal to the size of theeye's entrance pupil 210 when the exit pupil is located at the entrancepupil of the eye. In this figure, two exit pupils are used to create aseamlessly enlarged effective eyebox in the vertical direction. Thenumber and geometric arrangement of multiple exit pupils created at theeye can be widely varied and a large number of combinations arepossible. FIG. 6 is an illustration of three possible arrangements madewith multiple exit pupils 502. As more exit pupils 502 are added withadditional source wavelengths, the eyebox 501 covered by the multipleexit pupil's coverage improves approaching that of a larger syntheticexit pupil 601. Several example arrangements are shown for emphasizingparticular locations on the screen such as the center and corners (3, 5and 7 positions). Although small and spatially separated exit pupils 502are shown, the corresponding eyebox 501 for each exit pupil—shown aslarger black circles—will be larger than the individual exit pupils androughly the size of the eye's entrance pupil. If the individual eyeboxesassociated with each exit pupil location overlap one another the overalleffective eyebox can be made continuous without gaps in the viewableimage space. Individual exit pupils are shown as small circles, but canrange in size depending on the size of the scanned beam. The inventionincludes but is not limited to these arrangements. For larger field ofview systems, the most desirable solution may be a large number of exitpupils to fully cover the desired eyebox, i.e., 5 and 7 exit pupils inFIG. 6. To maximize the fill factor of the eyeboxes, the different exitpupils should be arranged within a triangular lattice. Alternatively,simpler solutions can be found by explicitly selecting which parts ofthe human vision and eye rotation the multiple exit pupils cover. Forexample with 2 exit pupils, a single corner and the center of the eye'srotation can be used. This would be useful for displaying information inone corner of the person's field of vision viewable with both foveal andperipheral vision. The exit pupils can be non-overlapping in field ofview or they can be overlapping.

Volume holograms can be effectively wavelength multiplexed provided thathologram is selective enough to prevent crosstalk between adjacentchannels. The selectivity of the hologram can be tuned controllingrefractive index variation of the material and material's thickness.This selectivity applies to both the wavelength of the reconstructionlight, and also the incidence angle of the reconstruction light withrespect to the recording light. In the design of a multiplexed head worndisplay there is subsequently a trade-off between selectivity andtolerance to spectral and angular misalignment between the hologramrecording and the readout when used as a head worn display. FIG. 7,showing the diffraction efficiency of holographic transflectors inordinate versus wavelength in abscissa as calculated by an analyticmathematical model, is an illustration of the wavelength selectivity ofa volume holographic element made in a photopolymer. In this examplethree holograms are multiplex recorded at three center wavelengths inthe red: 643 nm—reference sign 701—, 658 nm—reference sign 702—and 671nm reference sign 703—using a modeled refractive index variation of0.0075 and a 60 μm hologram thickness. The recorded holograms show highwavelength selectivity 704, 706 and 706 for low crosstalk readout. Inthis simulation two plane waves at 0° and 45° from each side of thehologram record the hologram. The hologram is subsequently read-out at45°. The refractive index considered is 1.5. From this modeled examplewith the three red wavelengths chosen, the crosstalk is approximately16% for the 16 μm photopolymer, but reduces to <1% for the thicker 60 μmphotopolymer. In yet another embodiment of the invention, the displaycan be made full color by using red, green and blue light sources. For afull color display, the required number of individually controlled lightbands is 3× the number of desired exit pupil locations. For example, 3light bands for one exit pupil and 21 light bands for 7 exit pupils.

In at least one embodiment of the invention, the holographictransflector is recorded using an optical holographic recording setupshown in FIG. 8. This setup creates a reflection hologram on the curvedHOE. FIG. 8 is a schematic of the holographic writing setup shown withthree wavelengths of similar color but different center wavelengths 801,802 and 803 for three exit pupil locations 829 relative to a holographicmaterial 824 collocated with the eye in the head worn display. Afterpassing through half wave plates 804, 805 and 806, the beams from thelasers are split with polarizing beam splitters 807, 808 and 809 andcoupled by focusing optics 813, 814, 815 and 816 into fibers 817, 818,819 and 820. The object beams 830 pass through lenses 827 and 825, arereflected by mirror 828, and pass through a polarizer 826 before passingthrough the holographic material 824 and focusing at the exit pupillocations 829. The reference beams 831 pass through lenses 821 and 823,a polarizer 822 and are reflected by a mirror 832 before passing throughthe holographic material 824. In the reference beams 831, all laserwavelengths are combined and presented to the HOE using the angles ofincidence and positions of the scanning projector. In the object beams830, the lasers are coupled into individual optical fibers, which arespatially arranged and relay imaged through the HOE to create themultiple exit pupils at the eye location. Although three laser sourcesare shown, the setup can be scaled for additional wavelengths. Therecording setup of FIG. 8 can be used as a “simultaneous” recordingsetup, where the multiplexed holograms are written at the same time.Simultaneous recording has advantages in terms of holographic efficiencyhomogeneity and a one-shot recording schedule. Other embodiments of theinvention could alternatively use sequential recording to multiplexholograms, where multiple recordings are done one after another. Anothermethod is to record each holographic film with one wavelength or a setof wavelengths and subsequently place the films on top of each other.

In at least one embodiment of the head worn display light sources ofdifferent wavelengths are beam combined coaxially for projection fromthe scanning projector. FIG. 9A is a schematic of a diffractive methodfor beam combining multiple LED or light sources of different centerwavelengths 902, 903, 904, 905 and 906. These sources may be containedwithin a single controlled source 901. A lens 907 focuses laterallyseparated beams before reflecting from a diffractive element 908, whichserves to co-align the beams of different wavelength before beingfocused by a lens 909 for a collimated output 910. FIG. 9B is aschematic of a dichroic beam combiner for collimated light sources ofdifferent center wavelengths 912, 913, 914, 915 and 916. Collimatedbeams from light sources of different wavelengths are co-aligned usingdichroic beam splitters 917 for a collimated output 918. FIG. 9C is aschematic of a volume holographic beam combiner for collimated lightsources of different wavelengths. Light sources of different wavelengths919, 920, 921, 922 and 923 are coaligned by a volume holographic element924 for an axially combined beam 925. FIG. 9D is a schematic of a volumeholographic beam combiner using an external cavity to create amultiwavelength laser source. Each of the sources 926, 927, 928, 929 and930 has an antireflection coating to prevent lasing within its internalcavity, and incident onto a holographic beam combiner 931. Instead, anexternal partially reflective mirror 932 forms a cavity for all sourcesfor collimated output 933. The advantage to this approach is decreasedsensitivity to temperature variations.

In at least one embodiment, the present invention relates to a head worndisplay with wavelength separated light sources that are combined in anon-coaxial way to create relative angle shifts between light from eachsource after leaving the scanning micromirror. Light from independentsources with appropriately separated wavelengths can be multiplexed inwavelength in the holographic transflector to produce spatiallyseparated exit pupils (and equivalently spatially separated eyeboxes) atthe eye. If light from the sources is coaxially combined, thensignificant preprocessed digital image shifts are required to align theapparent images from the different exit pupils at the eye. However, ifadditional angular separations are used to replace most of the largepreprocessed image shifts that were necessary to align the apparentimage at the eye by creating the shifts optically, digital preprocessedimage shifts can be mostly and potentially completely eliminateddepending on the optical correction of the projection optics andprecision of the optical alignment. With imperfect correction of opticaldistortion or imperfect alignment, the images of the different exitpupils can be aligned using small relative shifts and relative imagewarps (pre-compensated distortion correction) to form a well-alignedsingle apparent image to the eye. This approach has the advantage ofsaving pixels and image area, which are lost when significantly largepreprocessed image shifts are required. FIG. 10 shows a scanningprojector based head worn display on an eyewear frame 1001 that createstwo exit pupils 1005 and 1006. In this illustration both exit pupils arecontained within a single large entrance pupil 210 of the eye 209. Twoindependent sources of different center wavelengths are scanned byscanning projector 1002 creating shifted beams, which are reflected by amultiplexed holographic transflector 204 to the eye 209. The chief rays1003 and 1004 of the two sources are shown in solid and dashed lines forthe two wavelengths. Three mirror scan positions are shown representingthe center and extremes of the scan range. Any misalignment presentbetween the two images caused by slight misalignments or differences inthe distortion created by the optics can be corrected by independentpre-processing of the images. Although shown for two wavelengths intwo-dimensions for simplicity, this arrangement can be directly extendedfor additional exit pupils (wavelengths) with a three-dimensionalarrangement.

In at least one embodiment, the source combiner to create non-coaxiallycombined light sources consists of a spatially distributed ensemble oflight sources and a telecentric combiner lens which combines the sourcesat the exit pupil of the telecentric lens which coincides with thescanning micromirror. FIG. 11 shows an optical apparatus to combine themultiple wavelength sources with predetermined angles to present them tothe hologram. In this illustration two sources are shown, 1101 and 1102of different center wavelengths. The sources are accurately placed andarranged on a single plane such that their spatial positions aretransformed into angular separations by a telecentric lens 1103 whichhas its exit pupil collocated with the scanning micromirror 1104. Afterreflection from the micromirror 1104, light 1105 travels to a projectionlens (not shown) to focus the light for presentation to the hologramtransflector such that the light is collimated or nearly collimated atthe eye. The angles between the separate sources are designed such thatthe preprocessed image shifts necessary to align the different imagesfrom each exit pupils at the eye is minimized. The sources are arrangedwith precision alignment such that their orientation creates thenecessary angular separation after leaving the micromirror to minimizeimage processing shifts. These sources can be arranged in a variety oforientations including but not limited to those shown in FIG. 12. FIG.12 shows four possible source arrangements that could be used with theapparatus of FIG. 11. Each source emitter 1201 in this case can besimilar in apparent colour, red for example, but separated in wavelengthfrom its neighbours; or a combination of red, green and blue as shown inFIG. 13A. In at least one embodiment, the sources can be surface mountemitters such as but not limited to vertical cavity surface emittinglasers (VCSELs), LEDs, Superluminescent LED's (SLED's) or resonantcavity LEDs (RCLEDs) of different wavelengths which are aligned withprecision in a single electronic package using a precision pick andplace method. In another embodiment, broadband sources are created andspatially arranged using a monolithic fabrication method and thenmodified to narrow their bandwidth using wafer scale spectral filtersand optics. Further, in at least one embodiment, a full color exit pupilis created by closely spacing red, green and blue sources for each exitpupil location in a single electronic package. FIG. 13A shows anembodiment of the invention to extend the angularly shifted source ofFIG. 12 to full colour. An emitter ensemble 1202 is shown with threeseparated emitter positions 1301, 1302 and 1303 as in the far leftexample in FIG. 12. For each emitter positions, red, green and blue(RGB) emitters are arranged closely to form approximately a singleemitter position. As an illustration, the emitter position 1301 isconstituted of red emitter 1304, green emitter 1305 and blue emitter1306. Additional RGB wavelengths form the other separated emitters 1302and 1303 to ultimately create a three-exit pupil composite eyebox. Allemitters in this example should be separated in wavelength from eachother to allow for independent control and low crosstalk in themultiplexed holographic transflector. FIG. 13B shows how the slightlyshifted RGB emitters for a single eyebox create slightly angle shiftedscans at the eye, producing the image 1307 on the retina. To combinethese images for the human viewer, image preprocessing can be used toalign the images into a single apparent color image with an expandedcomposite eyebox 1305. In another embodiment red, green and blue lightsources are combined using a dichroic beam combiner. FIG. 14 shows theoptical apparatus to non-coaxially combine the multiple wavelengthsources from three panels 1401, 1402 and 1403 using a prism dichroicbeam combiner 1408. This optical arrangement is useful in particular forcombining red, green and blue (RGB) panels. In this example embodiment,sources 1101, 1102, 1404, 1405, 1406 and 1407 are shown in aconfiguration with two sources for each panel at different centerwavelengths. The sources are accurately placed and arranged on a singleeffective plane such that their spatial positions are transformed intoangular separations by a telecentric lens 1103 which has its exit pupilcollocated with the scanning micromirror 1104. After reflection from thescanning mirror 1104, light 1409 travels to a projection lens (notshown) to focus the light for presentation to the hologram transflectorsuch that the light is collimated or nearly collimated at the eye. Theangles between the separate sources are designed such that thepreprocessed image shifts necessary to align the different images fromeach exit pupils at the eye is minimized.

In yet another embodiment, light sources with relatively large spectralbandwidths are used; for example LEDs. To prevent crosstalk from theholographic transflector, spectral filtering may be required to reducethe bandwidth of each emitter. In at least one embodiment this can beachieved using a spectral filtering section such as but not limited tothe implementation shown in FIG. 15. In this arrangement, the sourcearray constituted of sources 1501 and 1502 would be as describedpreviously in FIG. 12 or FIG. 13, however this source would be firstreimaged onto a spatial filter mask 1505 after double passing through alens and reflecting from a diffraction grating 1504. By jointlyoptimizing the source arrangement, lens, diffraction grating and mask,light can be efficiently produced at the mask in the spatial arrangementneeded for the non-coaxial combiner shown on the left hand side of FIG.15. Further, in at least one embodiment, the light sources can be madeoversized-larger than the apertures in the filter mask. The purpose ofthis would be to reduce the alignment sensitivity of the opticalassembly in exchange for reduced power efficiency. FIG. 15 shows anoptical apparatus to non-coaxial beam combine and project which includesthe angular combiner of FIG. 11 along with a spectral and spatialbandwidth filter section. Alternatively, the spectral and spatial filtersections could be applied to the RGB combiner of FIG. 14. The spatialand spectral bandwidth sections prevent crosstalk when more broadbandsources such as LEDs and resonant cavity LEDs are used. Light sourcesseparated in wavelength and position 1501 and 1502 project to a lens1503 and then to a diffraction grating 1504 to disperse light from eachemitter in angle and then filter both spatially and spectrally theconjugate image using an aperture mask 1505. Following the aperturemask, the angular combiner of FIG. 11 combines the beams as describedpreviously. The light source implementation shown in FIG. 15 can becompactly implemented into eyewear by mounting the assembly within thearm of the eyewear. FIG. 16 shows a conceptual monocular HMD arrangementusing the angular source combiner and spatial and spectral filters 1601of FIG. 15. This arrangement also includes the holographic transflector1602 and the projection lens 1603. In another embodiment, FIG. 17 showsa conceptual binocular HMD arrangement using two of the angular sourcecombiners and spatial spectral filters 1601 and 1701 of FIG. 15. Thisarrangement also includes two holographic transflectors 1602 and 1702 oneach spectacle lens and two projection lenses 1603 and 1703. In anotherembodiment, the spectral and spatial filtering implementation can bedone in a transmissive rather than reflective arrangement, such as butnot limited to the implementation shown in FIG. 18. Compared to FIG. 15,the optical configuration of FIG. 18 is not a folded double-pass with areflective grating, but instead an in-line transmission arrangement.Light sources 1501 and 1502 separated in wavelength and position projectto a lens 1801 and then to a transmission diffraction grating 1802 todisperse light from each emitter in angle and then filter both spatiallyand spectrally when focused by a lens 1803 to the conjugate image on theaperture mask 1505. Following the aperture mask, the angular combiner ofFIG. 11 combines the beams as described previously. Further, in yetanother embodiment of the light source emitter with spectral filtering,a multiplexed reflection holographic element can be used to performspectral filtering on the light emitted from broadband sources beforereflecting from a scanning mirror, such as but not limited to theimplementation shown in FIG. 19. In FIG. 19 light sources separated inwavelength and position 1501 and 1502 project to a lens 1901 and then tothe multiplexed reflective volume hologram 1902. Diffracted beams fromthe HOE 1902 are filtered spectrally by the recorded volume hologrambefore being transported to the scanning mirror 1104 and then to anaperture filter 1903. The direction of the light can also be adjusted bygrating slant.

In at least one embodiment, the present invention of multiple exitpupils created with a holographic element relates to a head worn displaycreated using a microdisplay panel rather than a scanning mirrorprojection system. The panel microdisplay is illuminated by a sourcesuch as an LED or a laser light source and projected toward the eyeforming a conjugate image of the microdisplay on the retina. Thisholographic transflector element performs two primary functions. Itallows ambient light from the environment to pass through providingnormal vision to the user. It also redirects scanned light from theprojector to the eye to provide an image on the retina. In the presentinvention, the introduction of multiple small beams forming a compositeexit pupil reduces the étendue of the light beams compared to prior art(see FIG. 2, U.S. Pat. No. 4,940,204). This has two significant effects:

-   -   1) it reduces the size of the projection optics, making the HWD        smaller and lighter. This can be seen by examining FIG. 2, and        considering the effect of a smaller numerical aperture for the        light from the projector to the hologram;    -   2) it reduces the optical aberrations of the system as reducing        numerical aperture improves optical performance.

In published literature related to U.S. Pat. No. 4,940,204 it wasdescribed that despite significant efforts to optimize the hologramrecording and projection optics, the system did not perform to a highstandard due to aberrations which reduced image quality. By dramaticallyreducing the beam size (for example, by 10×) these aberrations can bebetter controlled for a high quality image. This requires that theeyebox be expanded, which is accomplished by wavelength or shiftmultiplexing to create a multiple-beam exit pupil at the eye location asdescribed above. In at least one embodiment, a field sequential colorLCOS microdisplay is used to project the different wavelengths of lightonto the hologram to create multiple exit pupils. Sources of differentcolors are combined and presented to the microdisplay. The microdisplaycycles through the colors sequentially such that the time to project allthe wavelengths constitutes one video frame. This requires that themicrodisplay refresh at a rate at least equal to (the number ofwavelengths)*(the desired video rate). The microdisplay then displays ashifted and distortion corrected image for each wavelength such that theimages are combined to form a single image at the eye due to persistenceof vision (see FIG. 4). Further, in at least one embodiment, a simpleeye tracking system is used to preferentially activate the exit pupilbest aligned to the eye. This has the advantage of reducing the requiredrefresh rate of the microdisplay and power consumption of the lightsources when using field sequential color since only the subset ofwavelengths for a single exit pupil needs to be displayed at any momentin time. In at least one embodiment, the projection optics used totransfer light from the microdisplay to the hologram use tilted,shifted, aspheric and non-rotationally symmetric optical elements topre-compensate for aberrations formed by the off-axis projection and thehologram's reflection. Similarly, the hologram recording setup (shown ina simplified configuration in FIG. 8) will utilize tilted, shifted,aspheric and non-rotationally symmetric optical elements to optimize thehologram for readout in the HWD with low optical aberrations. The jointoptimization of the projection optics and hologram recording reduces theaberrations of the arrangement to form low aberration and high qualityimages on the retina. In yet another embodiment, a light-emitting panelsuch as an OLED is used. Here the pixel structure-whether stacked orside-by-side for different emission colors-creates the multiplewavelengths necessary to create the multiple exit pupils at the eye.Each set of similar wavelength pixels is individually controlled toproduce a properly shifted and distortion pre-compensated image forprojection onto the hologram and subsequently to the user's eye forminga single apparent monochrome or color image using the differentwavelengths. FIG. 20 shows the multiple exit pupil arrangement with apanel microdisplay rather than scanning projector. Light from amicrodisplay 2002 within a projection module 2001 is projected throughprojection optics 2003 onto a multiplexed holographic screen 204. Theholographic screen 204 is multiplexed in wavelength to separate eachwavelength to create multiple exit pupils as also described in FIG. 3thru FIG. 7. A single image is created at the eye by preprocessing theimages as shown in FIG. 4. The projection optics 2003 consist of anarrangement of optical elements necessary to present light to theholographic element 204 such that the holographic element reflectsnearly collimated light toward the eye with low aberrations. Theseoptical elements of the projection lens consist of, but are not limitedto lenses, mirrors, freeform elements, shifted and tilted elements,aspheric lenses and mirrors, non-axisymmetric lenses and mirrors andprisms. Three light field positions 2004 are shown projected from theprojector. After reflection from the holographic reflector 204, threeangularly separated beams 2005, 2006 and 2007 form spatially separatedexit pupils at the eye 209.

In another embodiment of the multiple exit pupil head worn display, thepresent invention relates to system and method for placing two or moreexit pupils on the eye with control over their locations such that

-   -   1) at least one exit pupil provides an image overlaid with        on-axis see-through vision sub-display;    -   2) at least one exit pupil provides a independent and separate        image in a “glance-at” sub-display substantially off-axis from        normal vision. In at least one embodiment, the HOE is divided        into two or more areas representing separated fields of view,        with at least one area redirecting light in the exit pupil        providing the on-axis see-through sub-display, and at least        another area redirecting light in the exit pupil providing the        glance-at off-axis sub-display.

FIG. 21A shows an optical schematic of the scanned retinal display withthe eye aligned to one of two separated exit pupils with separated fieldof view created with a single wavelength beam 2101. The singlewavelength beam is incident on the scanning mirror 205 which divides theoverall field of view into the on-axis see-through display light 2104and the glance-at off-axis light 2105. Both sets of beams reflect fromthe holographic reflector 204 forming converging light bundles 2102 and2103 which form the two exit pupils at the eye 209. In this figure theeye line of sight 211 is aligned to the on-axis see-through light 2102.FIG. 21B shows the same system as FIG. 21A but with the eye line ofsight 211 aligned to the glance-at off-axis light 2103. In anotherembodiment, the usable field of view for both the on-axis see-throughsub-display and the off-axis glance-at sub-displays overlap partly. FIG.21C shows an optical schematic of the scanned retinal display with theeye line of sight aligned to one light bundle 2108 with partlyoverlapping field of view from a second light bundle 2109 created withtwo wavelength beams 2106 and 2107. Recording and reading the HOE bydifferent wavelengths or sets of wavelengths makes the differentiationbetween the on-axis see-through sub-display and the off-axis glance-atsub-display. In yet another embodiment shown in FIG. 21D, both theon-axis see-through sub-display field of view and the off-axis glance-atsub-display field of view can be overlapped allowing the user to viewalerts in the off-axis glance-at sub display while looking straightahead. In at least one embodiment, the MEMS scan mirror projects lightover a broad enough angular range to display information for both theon-axis see-through sub-display and the off-axis glance-at sub-display,as shown in FIG. 21A-FIG. 21D. Both the position at which the lightimpinges on the HOE and the light wavelength discriminate in which exitpupil the light is redirected, thus for which sub-display the light issent to. In at least one embodiment, a plurality of light sources—onefor each sub-display—are placed at different positions. FIG. 21E showsan optical schematic of the scanned retinal display with the eye alignedto one light bundle 2114 with a separate field of view from a secondlight bundle 2115 created with two light sources separated in positionand angle 2112 and 2113. Light generated from one particular positionimpinges on the HOE so as to be used for the on-axis see-throughsub-display, while light generated at another position impinges on theHOE so as to be used for the off-axis glance-at sub-display. In anotherembodiment shown in FIG. 21F, a flip mirror 2120 modifies the incidentangle of the light impinging on the scanning mirror 205, thus modifyingat which position light beams impinge on the HOE. Light can then be sentto the area providing an off-axis glance-at sub-display for a givenposition of the flip mirror, or be sent to the area providing an on-axissee-through sub-display.

In another embodiment of the invention, multiple scanning mirrors createmultiple exit pupils at the location of the eye by reflecting from aholographic optical element. In at least one embodiment the hologram isangle multiplexed for the multiple exit pupils. FIG. 22A is a top viewoptical schematic of the scanned retinal display with two scanningmirrors (only one visible) 2201, where light from two different anglesreflects from the hologram 204 back to the eye 209 to create twoseparated exit pupils at the eye. FIG. 22B shows the side view of thesame system of FIG. 22A where the two scanning mirrors 2201 and 2202create two separated beams 2203 and 2204 which in turn create twoindependent exit pupils at or near the eye's location. FIG. 23 shows asetup to record the angularly multiplexed holograms for the multipleexit pupil head worn display of FIG. 22A. This figure is similar in manyrespects to the wavelength-multiplexed embodiment of FIG. 8. Each laserused is split into two beams, which would interfere into the holographicfilm. On the reference arm, each light source send light to the hologramfrom distinct locations, resulting in different incidence angle at everyposition of the holographic film. For a selective enough holographicmaterial and distant enough incidence angles, each beam incident on aposition of a then recorded HOE is stirred into its correspondingeyebox. In FIG. 23 three lasers of the same center wavelength 2301 areused to produce a holographic transflector with three exit pupilscollocated with the eye in the head worn display. After passing throughhalf wave plates 2302, the beams from the lasers are split withpolarizing beam splitters 807, 808 and 809 and coupled by focusingoptics 2303 into fibers 2304. The object beams 2306 pass through lenses827 and 825, are reflected by a mirror 828, and pass through a polarizer826 before passing through the holographic material 824 and focusing atthe exit pupil locations 829. The reference beams 2305 pass through apolarizer 2307 prior the holographic transflector 824. In the referencebeams 2305, the lasers are presented to the HOE using the angles ofincidence representative of the multiple scanning mirrors. In the objectbeams 2306, the lasers are spatially presented to the HOE to create themultiple exit pupils at the eye location. Although three laser sourcesand multiplexed angles are shown, the setup can be scaled for additionalmultiplexed holograms.

In another embodiment of the invention, two collimated or nearlycollimated beams incident each side of the HOE interfere to produce areflective hologram for a non-pupil forming head worn display. The headworn display is referred to as non-pupil forming since no intermediateimage is formed in the optical system. A consequence of this type ofarrangement is that field of view is limited, and more difficult toincrease than the eyebox. With a non-pupil forming system the field ofview can be increased using two or more sets of collimated or nearlycollimated beams incident on opposite side of the holographic filminterfere on the holographic film to produce a reflective hologram asshown in FIG. 24A. Each set is constituted of two beams previously splitfrom the same coherent light source. In at least one embodiment, the HOEis multiplexed angularly. FIG. 24A shows three sets of beams. Thereference beams 2407, 2408 and 2409 are collimated by a lens 2406 andincident on the holographic material 2405. The object beams 2401, 2402and 2403 are collimated by a lens 2404 and incident on the opposite sideof the holographic material 2405. The then recorded holographictransflector will produce exit pupils whose location 2410 will be belowthe holographic material. In at least one embodiment, it is possible togreatly reduce the cross-talk by matching read-out and diffracted anglesof the different multiplexed HOE. In FIG. 24B, the diffractionefficiency 2413 and 2414 in function of wavelength are shown for twodifferent holographic transflectors as calculated by an analyticmathematical model. The diffraction angles 2411 and 2412 of the twoholograms as a function of incidence angle are also shown. Because theholograms are closely matched in their incidence angle to diffractedangle, they can be combined for a contiguous overall field of view. Thediffracted angle is of 3.304° for both the holographic transflectorrecorded with a reference beam angle of 45° and the holographictransflector recorded with a reference beam angle of 55°. In anotherembodiment, the non-pupil forming HOE is multiplexed spectrally ratherthan in angle. FIG. 25 shows the reflection hologram writing arrangementwith three multiplexed wavelengths. The object beam is made up of threespatially separated sources 2501, 2502 and 2503 of three differentwavelength which are collimated by a lens 2504 and brought together onthe holographic material 2505. The reference beam contains the threewavelengths beam combined into a single recording beam 2407, which iscollimated by a lens 2506, and then incident on the holographic material2505. The then recorded holographic transflector will produce exitpupils whose location 2508 will be below the holographic material.

FIG. 26 shows a drawing of a scanning projection monocular head worndisplay on an eyewear frame 2601. Light emitted from a light source 2604is projected by scanning mirror and focusing optics 2603 before beingreflected by a transflector element 204 to the eye 209. Display light isfocused on the retina 2602.

In another embodiment of the invention, confocal images of the eye aremeasured using light reflected from the eye and captured in a detectorwithin the scanning projection system. FIG. 27 describes an imagingsystem based on a confocal microscope arrangement. Collimated light fromthe light source 2709, which contains one or more light sources, isfocused by a first lens 2706 on a pinhole 2705. Light from the pinholeis collimated by a lens 2704 and scanned by a scanning mirror 2703 intwo dimensions. The light is directed and passe through a lens 2702 toan appropriate holographic transflector 204, which can be a volumeholographic element, a diffractive optical element (relief) or a curvedreflective element with frequency selective response. For a given angleof the scan, the light is focused on one point on the retina 2602. Apart of this light is reflected back, either at the same wavelength(Rayleigh scattering), by fluorescence, Raman or by non-linearphenomenon (frequency shifted from the incident light). The beampropagates back 2701 and is reflected by the holographic transflector.In case of frequency shifted light, the holographic transflector has anappropriate reflection at the shifted frequency. Since the image of thepinhole is the point on the retina where the light is focused, thepinhole will block light that is not coming from this focus. This is thebasic principle of confocal microscopy. It allows rejection of straylight and scattering light from other parts of space. After the pinhole,the reflected light is directed to a detector 2708 after reflection by abeam-splitter 2707. An image of the retina is thus formed point by pointby scanning the incident light beam with the scanning element. FIG. 28shows a variation of FIG. 27. The holographic transflector reflects theincident beam of a particular wavelength and focuses it 2801 in adifferent part of the eyeball (different than the retina). This way,other structures of the eye can be imaged. For example, light beam 2801can be focused on the crystalline lens of the eye. Some studies haveshown that the level of auto fluorescence (blue excitation) of thecrystalline lens is an indicator of diabetes. The wearable method allowsto continuously monitor the auto-fluorescence of the crystalline lensnon-invasively and thus provide valuable information to physicians. FIG.29 shows yet another variation. By using an additional wavelength in theinfrared, the visible head worn display is not perturbed (thetransflective screen does not modify visible light in this case). Imagesof the retina 2602 are obtained by one dedicated detector 2903 andimages of other parts (e.g. lens) are obtained with another detector2902 since the information is encoded spectrally in different color andthen detected separately. A diffraction type beam splitter is used toseparate the infrared and visible light 2901. FIG. 30 is anotherimplementation where the beam are separated, at the detector, by a beamsplitting filter 3001 which can be, but not restricted to, ofinterference type, dichroic, holographic. In another embodiment of theconfocal imaging method of the invention, a fiber optic is used in placeof the pinhole aperture as shown in FIG. 30. With an optical fiber 4301the light source is displaced to a different location from the scanningmirror, allowing for more flexibility in the eyewear design. The fiberitself is one from the following list: a single mode fiber, a multimodefiber or a dual cladding fiber, which includes both a single mode coreand a larger multimode core. With a dual cladding fiber the single modecore is used to project light into the HWD with high resolution, whilethe multimode core is used to collect the confocal return signal. Thisallows a larger aperture to be used for the confocal return signal forincreased signal to noise without sacrificing display resolution. Theabove embodiments are used to image the eye. Because the retina can beimaged, the veins can be identified and thus parameters such as the flowof blood cells, heart rate and arterial pressure (systolic, mean anddiastolic) can be measured using a number of ways. Heart rate can bemeasured, but not limited to, by using the small reflectivity changescaused by the varying blood volume occurring while the heart is pumpingblood. Because the imaging system is confocal, a precise region (such asa vein) can be isolated from the other part of the eye to extract thesignal with a high signal to noise ratio. Hemoglobin concentrationoscillates with time as a result of the arterial pulsation associatedwith the systolic-diastolic pressure variation. Thus a spectroscopicoptical measurement of the backscattered light from a vein is anothermethod to obtain arterial pulsation. The spectroscopic measurement canbe obtained by using two or more wavelength (such as for example twoavailable in the red of the HWD invention.

In another embodiment of the invention, scanned confocal images of theeye are used for eye tracking. FIG. 31 is an image of the retina takenwith a standard ophthalmoscope (microscope adapted for the eye). Theveins can be clearly seen. FIG. 32 illustrates the different parts ofthe retina that would be seen by an ophthalmoscope such as described inFIG. 27 to FIG. 30. The different images 3201, 3202 and 3203 correspondto different gaze directions. One can see that the images have featuresthat are clearly unique to gaze direction. Features can be extractedfrom the images that are unique markers and that can be used as uniqueidentifiers of the image representing a gaze direction and thus can beused as an eye tracker. The features are not restricted to images of theretina. Images from other parts of the eye, which move with eyedirection, can also serve as unique gaze identifiers. FIG. 33 is a blockdiagram describing the methodology for eye tracking: first the systemhas a calibration routine. The calibration could be done at intervals.The cycle depends on the rate of change of the features and adapted inconsequence. The calibration consists of grabbing images of the retinawhen the user performs different gaze directions. For example, the useris asked to fixate a feature displayed on the head worn display and thatmoves to cover a set of gaze directions. Features are then extractedfrom the images to form a reduced number of unique image identifiers. Adatabase of features corresponding to each gaze direction is formed. Byreducing images to unique features, less memory is used. Theneye-tracking is performed by first grabbing an image (retina or other)and then this image is correlated against the images in the database.Then a classification algorithm determines which image in the databasethe newly grabbed image most resembles. One method is to select theimage with the highest correlation. Another use of the anatomy of theretina is for relating intra ocular pressure (IOP) to optic discdisplacement. It has been shown that optical disc displacement isrelated to ocular pressure. Thus by monitoring the location of theoptical disc (region near the optic nerve in FIG. 31) continuously,information related to ocular pressure which is a major cause forglaucoma, can be extracted.

In another method, eye-tracking is realized by monitoring the intensityof the light reflected by the eye corresponding to each exit pupil. Inthe example of FIG. 34, showing 3 exit pupils, but not limited to, onlythe light corresponding to one exit pupil 3401 passes through the eyeentrance pupil 3402, the light corresponding to the other exit pupils3403 and 3404 is blocked. Hence the confocal imaging system will onlygive an appreciable signal from the exit pupil 3401 that is aligned tothe eye entrance pupil 3402. Because the relative spatial position ofthe eye-box is known in advance (calibrated), the relative intensityratio at the detectors (detectors in FIG. 27-FIG. 30) gives a measure ofthe gaze direction. FIG. 34 shows a front view of exit pupils arerelated to the “eyebox”.

FIG. 35 shows an implementation of a multiple exit pupil head worndisplay with a camera 3501 positioned to look directly in front of theuser. As shown in FIG. 36, the camera grabs a scene 3601 and provides azoomed version of the scene 3604 to the head worn display. For example,people with low vision, such as, but not limited to, cases of agerelated macular degeneration (AMD) for which there is a loss of visualacuity in the fovea but the peripheral vision is not affected canbenefit from having an image zoom option in order to better see detailsthat otherwise they would not be able to see in their macular region3602 without the zoomed digital image. This is accomplished by grabbingan image with the camera 3501, digitally processing the imageappropriately (such as zooming digitally, adding contrast to the imageor edge enhancement by way of example) and presenting the processeddigital image to the viewer field of view with the help of the augmentedreality wearable glasses display of this invention. In other low visioncases such as tunnel vision for which the subject has lot peripheralvision but maintains foveal acuity, the loss of peripheral vision can becircumvented by having a camera 3501 grab an image which is thenprocessed so as to extract useful information (such as objects outsidethe peripheral vision of the patient). This information is thendisplayed on the glasses in the foveal region so as to give the patientawareness of the surrounding he/she cannot see. In another embodiment,other sensors than a camera (which is sensitive in the visible orinfrared spectral region) are added to the wearable glasses of thisinvention such as, but not limited to, position sensors (incl.accelerometers), GPS sensors, pressure sensors or any type of sensorthat can locate the position in space of the wearable glass. Bycombining these positioning sensors with an eye tracking system such as,but not limited to, the system described in this application and byusing a binocular projection display, for example a system such as butnot limited to, with multiple exit pupil (one for each eye), one cangenerate augmented reality with 3 dimensional information. Thepositioning sensors locate the position of the eyewear with respect to ascene (e.g. recorded digitally by the camera). The eye tracking systemdetermines the object of interest in the scene. Thus with bothinformation, one can generate the appropriate augmented image to displayin relation to the object in the scene.

FIG. 37 illustrates embedded small kinoform mirrors 3701 and 3702 in atransparent matrix 3703 which can be any suitable thermo plastics (e.g.Poly Carbonate, PMMA). The function of the small mirrors is to redirecta direction of light beam from the projector to the eye 209. Theprojector can be a scanning mirror or a fixed microdisplay (LCD, LCOSe.g.). The mirrors can be coated with a thin film so as to provide anadequate spectral reflection response. For purpose of illustration,let's assume 3 eye boxes and monochrome operation. The spectral responseof the embedded mirrors is reflective in the spectral band of the 3light sources and transmissive everywhere else in the visible range. Byembedding the kinoform mirrors (which looks like a kinoform) in amaterial of the same index, the light passing through the holographictransflector is unperturbed.

FIG. 38 illustrates another method for obtaining multiple exit pupilsfor the purpose of synthesizing a larger eyebox. A broadband 3802 lightsource 3801, such as but not limited to light emitting diodes of smallemitting apertures (between 1 and 10 micrometers), resonant cavity lightemitting diodes of the same small emitting aperture, laser diodes, superluminescent diodes and vcsels, is first spectrally sliced 3803 toproduce a spectrum 3804 composed of discrete emission bands eachseparated by a spectral zone with no light. The resulting collimatedlight is then scanned by a 2D scanner 2703 which can be but not limitedto MEMs scanner, resonant or non-resonant, acousto-optic deflectors,liquid crystal deflector. A projection lens 2702 is used to produce adiverging beam 3805. A holographic transflector 204 recollimates thediverging beam according to the wavelength 3806 and 3807. For example,the holographic transflector can be but not restricted to, a holographicelement, a volume holographic element of the polymer, crystal or glasstype. A polymer holographic material is preferred since it can laminatedonto surfaces. Because the diverging beam is composed of multipledistinct wavelength bands (in the example of FIG. 38, there are twodistinct wavelength for illustration purposes), the reflection hologramhas a thickness and is made in such a way as to diffract wavelength band1 3807 to produce a collimated beam corresponding to a certain angularrange of the diverging beam. Similarly wavelength band 2 3806 isdiffracted to produce a collimated beam which is propagating in the samedirection as collimated beam of wavelength 1 but displaced spatially soas to produce an extended eyebox with multiple exit pupils.

FIG. 39 illustrates yet another method for obtaining multiple exitpupils. The same collimated light source 3801, as described in FIG. 38,is spectrally filtered 3804 and deflected by a 2D scanner 2703. Thelight beam is incident on a dispersive optical element 3901, such as,but not limited to, a transmission diffraction grating. After theprojection lens 2702, the transmission grating produces two distinctdiverging beams 3902 and 3903, corresponding to each wavelength band(two bands in the example illustrated in FIG. 39). The holographictransflector 204, of the same description as in FIG. 38, produces twodistinct re-collimated beams 3904 and 3905 corresponding to eachwavelength band, which forms the extended eye-box with multiple exitpupils.

FIG. 40 illustrates a multiple exit pupil projection system on each sideof a pair of glasses 2601 (binocular or biocular). Thus the left andright eye each receives an image from the respective projection system.By using the tracking system described in FIG. 32, FIG. 33, FIG. 34, butnot limited to, three-dimensional information can be displayed inaccordance with the gaze direction. Three dimensions comes frombinocular view. For example, a camera placed on the glasses 3501 grabsan image representing the view of the wearer. By calibrating the camerawith the viewer's gaze angle, it is then possible to augment the viewwith appropriate information. For example, in medicine, a surgeonwearing the eyewear, augmented information could be displayed directlyonto the surgeon's gaze direction. The position of important arteries,not seen directly by the surgeon's real view, could be overlayed.

FIG. 41 is an embodiment of a rigid structure on which the projectionsystem, i.e., source 2604, detector, scanner or microdisplay 2603 andholographic transflector 204 are placed on. The system is aligned on therigid structure 4101. The rigid structure is then positioned (attached)to the frame 4102 of an eyewear which can be, but not limited to aneyewear with prescription glasses. The holographic transflector isplaced between the projection system and the spectacle. The divergenceof the beam coming off the holographic transflector can be adjusted tocompensate for the user's eye prescription.

FIG. 42 is another embodiment with the holographic transflector 204placed on the outside of the spectacles 4101. Thus, the projectionsystem comprising light source 2604, detector, scanner or microdisplay2603 is attached on the side branch. The holographic transflector issecured such as but not limited to epoxied, clipped, screwed.

1-75. (canceled)
 76. A method comprising: emitting a plurality of lightbeams, a wavelength of at least one of the plurality of light beams todiffer from a wavelength of at least one other of the plurality of lightbeams; modulating an intensity of at least one of the plurality of lightbeams based at least in part on intensity information corresponding toan image to be projected; scanning the plurality of light beams in twodistinct axes to form the image; and redirecting the plurality of lightbeams to a plurality of exit pupils based at least in part on thewavelengths of each of the light beams to project the image at theplurality of exit pupils.
 77. The method of claim 76, wherein a firstexit pupil of the plurality of exit pupils is spatially separated from asecond exit pupil of the plurality of exit pupils.
 78. The method ofclaim 76, wherein the intensity of the light beams is representative ofpixel values within the image.
 79. The method of claim 76, comprising:forming a first bundle of light beams, each light beam of the firstbundle of light beams to have a wavelength substantially within a rangeof wavelengths of a first spectral band of color to be perceived byhuman vision; and redirecting each light beam of the first bundle oflight beams to a different one of the plurality of exit pupils.
 80. Themethod of claim 79, comprising: forming a second bundle of light beams,each light beam of the second bundle of light beams to have a wavelengthsubstantially within a range of wavelengths of a second spectral band ofcolor to be perceived by human vision; forming a third bundle of lightbeams, each light beam of the third bundle of light beams to have awavelength substantially within a range of wavelengths of a thirdspectral band of color to be perceived by human vision; redirecting eachlight beam of the second bundle of light beams to a different one of theplurality of exit pupils; and redirecting each light beam of the thirdbundle of light beams to a different one of the plurality of exitpupils.
 81. The method of claim 76, comprising applying, for each of thelight beams of the plurality of light beams, one or more of imageregistration or image distortion correction to the image to align theprojected image based on the spatial location of the plurality of exitpupils relative to each other.
 82. The method of claim 76, comprisingspatially arranging the plurality of exit pupils to form an enlargedeyebox for viewing the image.
 83. The method of claim 76, comprisingcombining the plurality of light beams coaxially to correct angulardifferences between the plurality of light beams corresponding to eachof the plurality of exit pupils.
 84. A system comprising: a lightsource, the light source to emit a plurality of light beams, awavelength of at least one of the plurality of light beams to differfrom a wavelength of at least one other of the plurality of light beams;a scanning mirror to scan the plurality of light beams in two distinctaxes to form an image; and a holographic optical element to receive thescanned plurality of light beams from the scanning mirror and toredirect the plurality of light beams to a plurality of exit pupilsbased at least in part on the wavelengths of each of the light beams toproject the image at the plurality of exit pupils.
 85. The system ofclaim 84, the light source to modulate an intensity of at least one ofthe plurality of light beams based at least in part on intensityinformation corresponding to an image to be projected;
 86. The system ofclaim 85, wherein the intensity of the light beams is representative ofpixel values within the image.
 87. The system of claim 84, wherein afirst exit pupil of the plurality of exit pupils is spatially separatedfrom a second exit pupil of the plurality of exit pupils.
 88. The systemof claim 84, the holographic optical element to form a first bundle oflight beams, each light beam of the first bundle of light beams to havea wavelength substantially within a range of wavelengths of a firstspectral band of color to be perceived by human vision, the holographicoptical element to redirect each light beam of the first bundle of lightbeams to a different one of the plurality of exit pupils.
 89. The systemof claim 88, the holographic optical element to forming a second bundleof light beams, each light beam of the second bundle of light beams tohave a wavelength substantially within a range of wavelengths of asecond spectral band of color to be perceived by human vision, theholographic optical element to redirect each light beam of the secondbundle of light beams to a different one of the plurality of exitpupils.
 90. The system of claim 89, the holographic optical element toform a third bundle of light beams, each light beam of the third bundleof light beams to have a wavelength substantially within a range ofwavelengths of a third spectral band of color to be perceived by humanvision, the holographic optical element to redirect each light beam ofthe third bundle of light beams to a different one of the plurality ofexit pupils.
 91. The system of claim 84, comprising an image processorto apply, for each of the light beams of the plurality of light beams,one or more of image registration or image distortion correction to theimage to align the projected image based on the spatial location of theplurality of exit pupils relative to each other.
 92. The system of claim84, the holographic optical element to spatially arrange the pluralityof exit pupils to form an enlarged eyebox for viewing the image.
 93. Thesystem of claim 84, comprising a combiner lens to combine the pluralityof light beams coaxially to correct angular differences between theplurality of light beams corresponding to each of the plurality of exitpupils.
 94. An apparatus comprising: a substrate; a plurality ofkinoform mirrors embedded in the transparent substrate to receive aplurality of light beams corresponding to an image to be projected; anda reflective layer disposed on the plurality of kinoform mirrors toredirect the plurality of light beams to a plurality of exit pupils toproject the image to the plurality of exit pupils.
 95. The apparatus ofclaim 94, the substrate a transparent substrate to transmit ambientlight through the substrate.
 96. The apparatus of claim 94, wherein thesubstrate is a polymer matix.
 97. The apparatus of claim 94, wherein theplurality of exit pupils are spatially separated from each other. 98.The apparatus of claim 94, comprising a head mounted device, wherein thesubstrate comprising the embedded kinoform mirrors is coupled to thehead mounted device.